.. _Tutorial control the size of a network Regression python:

Tutorial: control the size of a network
############################################

Impose the maximum number of links
====================================

Extract MetamaterialAntenna Example, see :std:ref:`Metamaterial Antennas test case`, provided with NeurEco installation. The created directory should contain the following files:

* x_test.csv
* y_test.csv
* x_train.csv
* y_train.csv

Once that's done, we will create import the needed libraries:

.. code-block:: python

  from NeurEco import NeurEcoTabular as Tabular
  import numpy as np

the next step is to load the training data:

.. code-block:: python

  x_train = np.genfromtxt("x_train.csv", delimiter=";", skip_header=True)
  y_train = np.genfromtxt("y_train.csv", delimiter=";", skip_header=True)

At this stage, we will build a regression model without controlling the size of the network (this will be our reference point for comparison):

.. code-block:: python

  builder = Tabular.Regressor()
  ''' Building the NeurEco Model '''
  builder.build(input_data=x_train, output_data=y_train,
                # the rest of these parameters are optional
                write_model_to="./MetamaterialsAntennas/MetamaterialsAntennas",
                checkpoint_address="./MetamaterialsAntennas/MetamaterialsAntennas.checkpoint",
                valid_percentage=33.33,
                outputs_normalize_per_feature=True,
                links_maximum_number=0)

Once that's done, we  will create a regression model where the size of the network is limited (for embedding). To control the size, we just need to change the parameter *links_maximum_number*. This is an integer that controls the maximum number of trainable parameters.

.. code-block:: python

  builder.build(input_data=x_train, output_data=y_train,
                # the rest of these parameters are optional
                write_model_to="./MetamaterialsAntennas/MetamaterialsAntennas_Mbed",
                checkpoint_address="./MetamaterialsAntennas/MetamaterialsAntennas_Mbed.checkpoint",
                valid_percentage=33.33,
                outputs_normalize_per_feature=True,
                links_maximum_number=150)

  ''' Delete the builder from memory'''
  print("Deleting the NeurEco builder".center(60, "*"))
  builder.delete()

Once the two models are created, we will compare their sizes and performances. We will load the testing data, and check the relative l2 error of each network on the unseen set of data. After which, we will compare the sizes of these two networks.

.. code-block:: python

  print("Loading the training data".center(60, "*"))
  x_test = np.genfromtxt("x_test.csv", delimiter=";", skip_header=True)
  y_test = np.genfromtxt("y_test.csv", delimiter=";", skip_header=True)

  neureco_model = Tabular.Regressor()
  ''' Loading the normal size model '''
  neureco_model.load('./MetamaterialsAntennas/MetamaterialsAntennas')
  print("Regular model n° of trainable parameters: {0}".format(neureco_model.vec.size))
  out_ = neureco_model.evaluate(x_test)
  l2_error_regular = neureco_model.compute_error(out_, y_test)
  print("Regular model l2 error on testing data: {0}".format(l2_error_regular))
  neureco_model.plot_network()

  ''' Loading the Mbed  model '''
  neureco_model.load('./MetamaterialsAntennas/MetamaterialsAntennas_Mbed')
  print("Embedded model n° of trainable parameters: {0}".format(neureco_model.vec.size))
  out_ = neureco_model.evaluate(x_test)
  l2_error_mbed = neureco_model.compute_error(out_, y_test)
  print("Embedded model l2 error on testing data: {0}".format(l2_error_mbed))
  neureco_model.plot_network()
  neureco_model.delete()

The output of the previous code is as follows:

.. code-block:: text

    Regular model n° of trainable parameters: 1361
    Regular model l2 error on testing data: 0.010744763022509518

    Embedded model n° of trainable parameters: 298
    Embedded model l2 error on testing data: 0.019175265003600478

The two networks architectures are given in the following figures

.. list-table:: 

  * + .. figure:: ../../../../images/MbedRegularRegressor.png
          :width: 400

          Regular network

    + .. figure:: ../../../../images/MbedEmbeddedRegressor.png
          :width: 400

          Embedded network

.. _Tutorial select a model from a checkpoint python:

Select a model from a checkpoint
=================================

We can also control the size of the network, by selecting any model from the checkpoint file of a built model, and restart the build from that stage. We can choose to enrich the model by adding some trainable parameters, or not (meaning that the weights will be optimized but without adding trainable parameters).
In this case, there are 3 building parameters to use:

* *checkpoint_to_start_build_from*
* *start_build_from_model_number*
* *freeze_structure*

For example, using the regular network built above we can choose the checkpoint n°5, and restart the build from that point.

.. code-block:: python

  new_model = Tabular.Regressor()
  index_of_model_to_improve = 4

  new_model.build(input_data=x_train, output_data=y_train,
                  # the rest of these parameters are optional
                  checkpoint_to_start_build_from="./MetamaterialsAntennas/MetamaterialsAntennas.checkpoint",
                  start_build_from_model_number=index_of_model_to_improve,
                  freeze_structure=True,
                  write_model_to = "./MetamaterialsAntennas/MetamaterialsAntennas_improved",
                  checkpoint_address = "./MetamaterialsAntennas/MetamaterialsAntennas_improved.checkpoint",
                  valid_percentage = 33.33,
                  outputs_normalize_per_feature = True)

  print("Imroved Model n° of trainable parameters: {0}".format(new_model.vec.size))
  new_model.plot_network()

The new model will have the following architecture:

.. figure:: ../../../../images/MbedNetworkFromCkpt.png
    :width: 450
    :alt: MbedNetworkFromCkpt
    :align: center

    Controlling the size of a network by restarting the build from a checkpoint